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Abstract —Compressed sensing (CS) is an emerging paradigm 
for acquisition of compressed representations of a sparse signal. 
Its low complexity is appealing for resonrce-constrained scenarios 
like sensor networks. However, snch scenarios are often coupled 
with unreliable commnnication channels and providing robust 
transmission of the acquired data to a receiver is an issue. 
Multiple description coding (MDC) effectively combats channel 
losses for systems without feedback, thus raising the interest in 
developing MDC methods explicitly designed for the CS frame¬ 
work, and exploiting its properties. We propose a method called 
Graded Quantization (CS-GQ) that leverages the democratic 
property of compressive measurements to effectively implement 
MDC, and we provide methods to optimize its performance. A 
novel decoding algorithm based on the alternating directions 
method of multipliers is derived to reconstruct signals from 
a limited number of received descriptions. Simulations are 
performed to assess the performance of CS-GQ against other 
methods in presence of packet losses. The proposed method is 
successful at providing robust coding of CS measnrements and 
outperforms other schemes for the considered test metrics. 

I. Introduction 

Compressed sensing (CS) H] is a novel theory for sam¬ 
pling and acquisition of sparse and compressible signals. The 
traditional paradigm based on sampling a signal according to 
the Nyquist/Shannon theorem followed by compression can 
be replaced by the acquisition of a small number of linear 
measurements, in the form of random projections. This is 
very appealing for low-complexity systems, such as low-power 
sensor motes, where classic acquisition followed by compres¬ 
sion could be expensive in terms of energy consumption and 
computational demands. Such systems typically need to trans¬ 
mit the acquired data to a receiver over unreliable channels, 
thus raising the issue of robustness of the transmission against 
channel losses and at the same time imposing a constraint 
on the complexity of the adopted methods. The framework 
of multiple description coding (MDC) allows to increase the 
robustness by creating multiple correlated representations of 
the data to be transmitted. The quality of the decoded data 
will then depend on the number of descriptions available at 
the receiver, where side decoders can recover a lower quality 
version of the data from few received descriptions, whereas 
the central decoder can achieve the maximum quality when 
all the descriptions are received. A few works studied the 
problem of MDC in the CS framework. An early work on the 
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topic ill borrows from the theory of sparse representations to 
derive a two-description coding scheme through a frame syn¬ 
thesis operator. However, the work does not consider random 
projections. The work in a presents a method to generate 
descriptions of an image by sensing each of them separately, 
i.e., preprocessing the image to split it into two sub-images 
and then sense the wavelet coefficients of each separately. We 
will not consider this kind of approach in the paper because 
we want to explicitly avoid any preprocessing of the signal 
before acquisition. Indeed, compressive measurements could 
be directly obtained by specialized hardware {e.g., a-ii), 
thus hindering any preprocessing of the data. Deng et ai, 
m argue that the democratic property of random projections 
makes compressive sensing image coding robust to channel 
losses. They just partition the measurements into packets so 
that the quality of the decoded image will depend on the 
number of received packets. However, they do not consider 
a practical packetization problem, as they employ very small 
packets containing few measurements for each transmission. 
Such system is actually not sensible due to the high packeti¬ 
zation overhead. 

This paper builds on the MDC mechanism called graded 
quantization (CS-GQ), originally proposed in 0, exploiting 
the democratic property of compressive measurements. The 
principle behind CS-GQ is that multiple copies of the measure¬ 
ment vector can be used as descriptions. Each description is 
then partitioned into sets of samples, and each set is quantized 
with different quality. The principle is similar to ||9l- llT9l . but 
has never been applied in conjunction with CS. 

In this paper, we give several novel contributions with 
respect to 0. First, we discuss how methods based on 
segmentation of measurements such as Q are special cases 
of CS-GQ and, contrary to the present literature on MDC 
for CS, we carefully consider packetization issues. The use 
of CS for low-complexity and low-energy sensor networks 
motivates us to pay particular attention to the use of the 
proposed method with common data link layer protocols such 
as IEEE 802.15.4. Moreover, we propose techniques to im¬ 
prove the performance of CS-GQ, to optimize its parameters, 
and we introduce a novel algorithm based on the alternating 
directions method of multipliers (ADMM) ll20l to implement 
the side decoder. On the theoretical side, perfomance analysis 
is conducted by providing reconstruction guarantees for the 
side decoder, using both the actual decoder and an ideal 
version based on the oracle decoder, which serves as a limit 
case for performance assessment. On the experimental side, 
simulations are performed for various scenarios. The paper 
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is organized as follows; Sec. |II] explains some introductory 
background concepts. Sec. Hill introduces CS-GQ and Sec. IIVI 
the side decoding algorithm based on ADMM. Sec. FVl presents 
theorems guaranteeing stable side decoding and discusses the 
performance of oracle-based side and central decoders. Sec. m 
deals with the optimization of the CS-GQ parameters as func¬ 
tion of the channel loss probability and discusses packetization 
issues. Finally, Sec. IVIII reports the results of simulations 
comparing various MDC methods, the performance of CS- 
GQ in presence of channel losses using two different channel 
models and shows that the robustness of multiple descriptions 
can improve scene recognition accuracy in a sensor network 
application. 


II. Background 

A. Compressed sensing 

In the standard CS framework, introduced in Cl, ED, a 
signal X G which has a sparse representation in some 

basis S i.e. x = 'I'0, ||6||o = k n can be 

recovered from a smaller vector of noisy linear measurements 
y = $x -f e, y S R"*^^ and k < m < n, where <1) G R"*^" 
is the sensing matrix and e G R"*^^ is the vector representing 
additive noise such that ||e ||2 < £, using ll22l 

0 = argmin ||6||i s.t. — y ||2 < e (1) 

0 

and X = known as Basis Pursuit DeNoising, provided that 
m = 0{k\og{n/k)) ||2T1 and that each submatrix consisting 
of k columns of <i)'I' is (almost) distance preserving ll^ 
Definition 1.3]. The latter condition is the Restricted Isometry 
Property (RIP). Formally, the matrix iPT' satisfies the RIP of 
order k if 3Sk G (0,1] such that, for any 6 with ||0||o < k: 

(1-4)||0||^<||$'&0||B(1 + 4)||0||^, 

where Sk is the RIP constant of order k. It has been shown 
in ED that when $ is an i.i.d. random matrix drawn from 
any subgaussian distribution and Ti is an orthogonal matrix, 
$111 satisfies the RIP with overwhelming probability. More¬ 
over, using a random matrix as sensing matrix, ensures the 
democratic property of compressive measurements y ll25l . 
Indeed, it can be seen that each measurement has roughly 
the same importance as the others, in the sense that none of 
them improves or degrades the quality of the reconstruction 
significantly more than the others. This property is the key 
ingredient of our proposed CS-GQ technique, as described in 
Section HII] 

B. Multiple description coding 

MDC ll26l is a way of coding an information source that is 
resilient to packet losses. The multiple description technique 
allows to create multiple correlated representations of the orig¬ 
inal information source, each carrying enough information to 
decode the source with a certain fidelity. Losing a description 
will not make the received data unusable since each description 
can be decoded separately, albeit providing a limited quality. 
However, the best decoding quality is obtained when all the 
descriptions are available. In the framework of MDC, one can 



Fig. 1: Block diagram of a two-description MDC system. 
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identify the so-called central decoder and the side decoders. 
The role of the central decoder is to decode the source 
when all descriptions have been received, thus achieving the 
so-called central distortion. The side decoder recovers the 
source with a lower fidelity (side distortion) since only a 
subset of the descriptions has been received. This scheme 
is depicted in Fig. [D for the simple two-description case. 
One may seek to create balanced or unbalanced descriptions 
depending on their individual contribution to the final quality 
of the recovered source. Several approaches to MDC have 
been studied in the literature. Among others, it is possible to 
identify approaches based on transforms, such as the Pairwise 
Correlating Transform ED, approaches based on channel 
coding, such as Unequal Error Protection ll28l and approaches 
based on quantization, notably the Multiple Description Scalar 
Quantizer (MDSQ) ifTTl . 

III. Graded quantization formulation 

In the simple case of two descriptions, CS-GQ partitions 
the indices of the measurement vector y into two sets Hi 
and kl 2 - Two different quantization step sizes (coarse and 
fine) are chosen and assigned to each set to generate the first 
description. The dual assignment is performed for the second 
description. This is graphically shown in Fig.|2l where we con¬ 
sidered Hi = {i G [l, y] } ^2 = {j G + but, 

thanks to the democratic property ||25]| . the same performance 
is expected for any other definition of the sets with the same 
cardinality, e.g., the sets of even and odd indices. Before going 
into further details, we remark that this scheme is amenable to 
generalizations to an arbitrary number of sets and quantization 
step sizes and hence of descriptions. In this paper we will not 
consider this more general case, and we will rather focus on 
the analysis and usage of the two-descriptions system. The 
proposed system design may provide either balanced or unbal¬ 
anced descriptions, depending on how many measurements are 
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Fig. 3; High and low resolution quantizers are staggered so 
that the central decoder achieves higher precision. 


quantized using the high-rate and the low-rate quantizers. In 
case of unbalanced descriptions, the cardinality of sets Hi and 
H 2 is not the same. Balanced descriptions offer the same side 
decoding distortion regardless of which description is received. 
Otherwise, unbalanced descriptions may have significantly 
different side distortion depending on the specific description 
that is received. This is desirable when the descriptions are 
sent on separate channels having very different packet loss 
probabilities. In the simplest case, one description is contained 
in a single packet, e.g., when the number of measurements is 
small or there is no packet-size limitation. Issues related to 
packetization strategies and channel losses will be discussed 
later in Section |VT] 

From now on, we consider staggered uniform scalar quan¬ 
tizers with 2® and 2^ levels, and B > b. The associated 
quantization step sizes are, respectively, Ab and A},, which are 
linked to the number of levels through the dynamic range r: 
Ab = r2~^ and Ab = r2~^. The i-th description, i = 1,2, 
has then mB,i elements quantized with 2^ levels and uib^i 
elements quantized with 2^ levels, such that mB,i + rnb,i = m. 

Staggered quantizers are quantizers whose reconstruction 
levels are shifted with respect to each other. In this work, the 
staggering involves the low-resolution and the high-resolution 
quantizers and is motivated by an improvement in the reso¬ 
lution obtained by the central decoder, which receives both 
the high-resolution and low-resolution versions of each mea¬ 
surement. In fact, when both descriptions are received, each 
measurement falls inside the intersection of the quantization 
intervals defined by the two staggered quantizers, which is 
smaller or equal than Ab- Hence, the low-resolution quantizer 
has its bins shifted by ■^. The following equations describe 
how the measurement vector y is quantized to generate the 2 
descriptions: 


y(B)^ 




Ab 


y-¥ 

Ah 


Ab 


2 



This allows to gain some extra bins of width when low- 
resolution and high-resolution quantizers are combined. This 
is shown in Fig. [3 for a 4-bit high resolution quantizer and a 
2-bit low-resolution quantizer. In general, the high resolution 
quantizer has 2® bins, the low resolution quantizer has 2^ bins, 
while the combined quantizer has 2^ — 2^ -f 1 bins of size A b 
and 2(2^ — 1) bins of size 

In this work we do not consider more complex quantizers, 
e.g., the Lloyd-Max method or vector quantization, because 


they are regarded as computationally too complex and with 
little or no performance gain, as shown in 1291 . 

An improvement over classic recovery from noisy measure¬ 
ments O can be obtained if we explicitly take into account 
the two different quantization noise levels. Hence, the side 
decoder solves the following optimization problem: 


ds = argmin ||0| 

0 


s.t. 


(B) _ $(-B)v[/0 


■(.B) _ (pC-Blq/g 

— 2 

A) _ ^ 

< £b 

A) _ (j)(6)\i/0 

11 00 

VI 


( 2 ) 


where and are the appropriate submatrices of 4), i.e., 
4) restricted to the rows corresponding to the measurements 
with fine and coarse quantization levels, respectively. The two 
^ 2 -norm constraints in (|2l) take into account the different quan¬ 
tization levels in the two sets, being eb and £b the expected 
norm of quantization noise, which can be estimated as 


£b 



and Eb 



The £oo-norm constraints enforce consistency with the quan¬ 
tization intervals - i.e., a reconstruction whose measurements 
would be quantized to the same intervals as the observed mea¬ 
surements, allowing better dequantization performance l25l . 
m thus exploiting all the information available regarding 
the original unknown signal. Note that, while the f 2 -nonti 
constraints refer to the average norm of the noise, the £ao-^oim 
constraint is applied individually on every linear measurement. 
In 0, it is shown that explicitly considering the previously 
explained structure of the noise in the reconstruction problem 
provides significant gains in the quality of the reconstruction, 
with respect to considering an average noise norm. 

On the other hand, when both descriptions are received, the 
central decoder outputs the vector 


0c = argmin IIs.t. ||yc - $^0||2 < ec , (3) 

0 


where yc is obtained in the following manner. For each 
measurement, the quantization bins of the high- and low- 
resolution versions are compared to determine the correct 
bin in the central partition to be used for the dequantization 
(see Fig. |3l. Eq can be estimated as in (fTsT i to properly take 
quantizer staggering into account. 

We shall also consider a special case of CS-GQ, which 
we call CS-SPLIT. It is a simple technique that consists in 
splitting the measurements vector into 2 or more segments, 
so that instead of transmitting a single packet with all the 
measurements, two descriptions are created by packetizing half 
of the measurements in each. Referring to Figure|2] CS-SPLIT 
can be regarded as a special case of CS-GQ when 6 = 0, 
providing the best central distortion but worst side distortion 
as shown in section IVI-AI 


IV. ADMM-based side decoder 

In this section, we propose a method for the resolution of 
problem (O based on ADMM. In 0 problem ([2]) is solved by 
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CVX 122 Il32). a framework to model and solve convex opti¬ 
mization problems, which employs the semidefinite program¬ 
ming (SDP) solver SeDuMi SeDuMi uses interior point 
methods to solve general SDP problems, resulting in rather 
slow performance, especially for large scale problems due to 
the fact that those are second-order method^. In this paper, we 
propose a significantly faster reconstruction algorithm based 
on the Alternating Directions Method of Multipliers (ADMM) 
tailored to solve problem (|2]i. ADMM ll20l is a popular first- 
order method that allows cheaper iterations, and that combines 
the robustness of the method of multipliers (a method to 
solve a constrained optimization problem via unconstrained 
minimization of an augmented Lagrangian function, including 
the constraints as penalty terms and an additional £2 penalty 
as augmenting term) ll?5l in terms of convergence, and the 
decomposition property, similar to dual ascent lf3^ . which 
also allows for a distributed implementation. It is particularly 
convenient to solve the side decoding problem (| 2 ]i because it 
allows to use alternating updates to the dual variables of an 
augmented Lagrangian, projecting over the sets defined by the 
multiple constraints. A proximal gradient method is used to 
update the primal variable 9 at each iteration. In the following 
we report the derivation of the algorithm, while the complete 
procedure is summarized in Algorithm [T] 


A. Notation 

Let us first introduce some notation that is used during the 
derivation of the algorithm. First, we define the sets C 2 and Coo, 
which are feasible sets defined by the £2 and constraints 
of (|2|i, as a function of generic quantities A, b and c, as 

C 2 (A,b,c) = { 0 GM" I ||A 6 >-b ||2 < c} 

Coo (A b, c) = {e G M" I \\Ae - b|loo < c} , 


along with the corresponding indicator functions, namely 


^C 2 (A,h,c) ( 0 ) 


0 if 0 G C 2 {A, b, c) 
00 otherwise 


1 -C^(A,h,c) (^) 


0 if 6/GCoo(Ab,c) 

00 otherwise 


The quantities defined above will be used to replace the 
constrained formulaton of problem (| 2 ]| with an unconstrained 
one. Moreover, we define the soft-thresholding operator with 
threshold A as 


5a (v) = sgn(v) © max { 0 , |v| - A} , 


where | • | denotes the vector containing the absolute values of 
• and 0 denotes the elementwise product, and the operators 
performing projection over £2 ball 


'PC2(b.c) (v) 


b+C||^^ if ||v-b||2>c 
V otherwise 


and £00 ball 


clip[_^_+^] (v) = sgn(v) ©min{|v|,c} , 

where min{-} and max{-} have to be intended as elementwise 
operators. 


B. Unconstrained problem and auxiliary variables 

It is possible to use indicator functions to transform (|2]i into 
an unconstrained minimization problem. First, we define the 
following quantities 

H = and h = 

L = $(«>) T- and 1 = 

Then, 

9s = argmm| ||6>||^ -f 2 c 2 ©B.ff.h) {d) + ,l,\) {0) 

+ ^Coo(eB,ff.h) (^) +1-C^(eb,L,\) (^) | 

= argmm| ||6>||^ -f 2 c 2 (£B.Lh) 

where I is the identity matrix of suitable size. We now 
introduce auxiliary variables w, z, p, q. 

w = H9, z = L9, p = H9, q = L9 

The problem can be now recast as: 

9 = arg min | ||6/||i + levies J,h) M (z) 

0,w,z,p,q 1. 

subject to w = H9, z = L9, p = iCS, q = L9 

We can include these new constraints by building an aug¬ 
mented Lagrangian functional: 

J = ll^lli +2c2(Ej3,7,h) (w) +Tc2(Ei,./.i) (z) (P 

+ (w - H9) -b (z - L9) 

+ (p - H9) H- U (q - L9) + ^||w - H9\\l 
+ P-\\^-L9\\l + ^\\v-H9\\l + ^\\ci-L9\\l . 

Moreover, we switch to the scaled form by defining the 
following residuals and scaled dual variables: 

r„=w—iCS ry=z — L9 rs=p — H9 rt = q—L9 

Up = Vp = Sp = p~^S tp = p~^t 

Hence, the augmented Lagrangian can be rewritten as: 

J = ||e||i + 2 :C 2 (eB. 7 .h) (w) +Tc2(e6./.l) (z) + ^P 

+ 2^C„„(%.7.i) + Up||^ - ^IIUpll^ 

+ + Vpll 2 “ ^Il'^pll 2 + + ®pll 2 

“ ^l|Spll 2 + + fpll 2 “ ^I|fpll 2 ■ 


'See 03 for details. 


( 4 ) 
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Algorithm 1 CS-GQ side decoder. 


Require: a, A, p, H, L, h, 1 


e 


( 0 , 0 ) 


^ 0 , ^ 0 , ^ 0 , <— 0 , v® = t® ^ 0 


while Outer stopping criterion is not met do 
while Inner stopping criterion is not met do 


e 




= Sx(9‘'"’^^ - ap + lF' 


W 


( 


- p(*) - s« 




LQi^j) _ q(*) _ t« 


J J + 1 

end while 

After stopping: 


Ai+l) 


.(*+ 1 ) = 


h + es 



(i+l.O) 


l|rf-0U+i.o)_uW_h||2 


if WHO 


(^+M)_uW_hn 


- Uo 


Otherwise 


^ 0 (i+i,o)_vW_l 
||L 6 >(*+U 0 )_vJ,‘)-l|| 


if ||^0(z+l.O)_^W_l||^>^^ 

Otherwise 

-s« - hi 


(i+1,0) 


— V 


(*) 


p(*+i) =clipj_^^^j 


') 

- fW - l) + ] 


u^*+i) = uW + 
v^*+i) = vW + z(‘+i) - 


[■(t+i) _ .(•(*) 


Ai+l) 


- L6» 


(*+ 1 , 0 ) 


i ^ i + 1 

end while 
Ensure: 9$ ^ 


(5) 

( 6 ) 

(7) 

( 8 ) 


C. Alternating directions minimization 

It is now possible to minimize (|4|i in an iterative fashion 
using alternating directions, i.e., minimizing over one variable 
at a time. This constitutes the outer loop of Algorithm [T] 


l(*+l) 

= 

argmjn| ||6||i 





-f 

fl|w“> 

'-7T6 + u«||2 

+ fl|z« 

-Le + 

^11^ 


-f 


-7^6 + s«||^ 

Ff||q« 

-L0 + 

11^ 







(9) 

.(*+1) 

= 

argnun{Zc2(eB,/,h) (w) 




+ 

P\\ 

— w - 

2 " 

- -f 

11 ^} 


(10) 

,(*+1) 

= 

argmm|Zc2(e6,/,i) (z) 





-f 

^||z- 

2 " 

2,6)(*+i) +vW|| 

*} 


(11) 

,(*+1) 

= 

argmm{l^^(^^h) 

(p) 




+ 

P\\ 

2 IIP- 

-f s^ll 

1 ?} 


(12) 


^(*+1) 

= arg 




-ill' 

q-L60+i( 

+ tS’lli} 

,(*+1) 

p 

= u(* 

p 

)+w0+O- 

- iF60+0 

.(i+l) 

P 

= v(*' 

p 

)+z0+O- 

/^5)(i+l) 

.(*+1) 

P 

_s(*) 

“p 

-f - 

iT60+i) 

.(i+1) 

'P 

-+(0 

+ q(*+i)_ 

^^(*+1) 


It can be seen that dUl involves minimizing a functional 
composed of two main parts: a smooth (differentiable) part 
f{e) = £||wW-i?e+u?||i + f||zW-L0+v«||l + f||p«- 
H6 + Sp II 2 + 2 ~ 7*6 + tp^lll whose gradient can be 

computed analytically and a non-smooth (non-differentiable) 
part g{9) = ||6||]^. Functionals of this kind can be minimized 
using the proximal gradient method llJTl . i.e., an iterative 
method that takes a step towards the negative gradient of the 
smooth component and then computes the proximal mapping 
over the non-smooth part. It is easy to show that the gradient 
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of the smooth part f{ 6 ) is: 

Ve/( 0 ) = 

= pH'^ +pL^ 

+ pH'^ +pL^ (L 0 (*)-qW-t«) , 

while the proximity operator of non-smooth part g{9) is the 
soft-thresholding function. Hence, one can iteratively take a 
step in the gradient direction and apply the soft thresholding 
function in order to eventually find i.e., the minimizer 

of (|9]l. This constitutes the inner loop of Algorithm [T] 

The subsequent minimization problems in ( fTOb . (fTTT i. (fTSl i 
and (foi l all involve projections over the sets dehned using 
the indicator functions. It can be seen that (fTOl) and 
correspond to projections over the £2 balls of radii eb and 
Sb centered at h and 1 respectively. Similarly, and (fTST i 
correspond to projections over the l^o balls of diameters 
As and Af, centered at h and 1 respectively. The results 
of the projections are ©, ®, 0 and ©. Notice that the 
projection operations and the update of the dual variables can 
be performed in parallel. 

As stopping criteria of the inner and outer loops, we check 
the distance between two successive iterations. Hence, two 
constants and are properly chosen such that the 
stopping conditions are 

|| 0 (*J + 1 ) _ 0 (*J )||2 

- 7— T- < (Inner stopping criterion) 

||g(i-|-l,0) _ giifi) 11 

- 7 - 77 ;- < (Outer stopping criterion) 


V. Theoretical guarantees 

In this section we present some theoretical results concern¬ 
ing CS-GQ. First, Theorem [T| provides a guarantee of stable 
recovery of signals from single descriptions, using the side 
decoder presented in 0. As it is common in the literature 
about CS (see for example ||2T| ). this kind of bounds is used 
as a guarantee that the error does not explode, rather than 
giving an exact characterization of the error itself 

Theorem 1. (Stable side recovery) Suppose that A = $'1' 
satisfies the RIP of order 2k with 62 k < \/2 — 1 and let y = 
AG + e where, without loss of generality, e = [e^ e^] with 
lleslU < £b, ||eb ||2 < £b, llesHoo < ||eb||oo < Then 
the solution to 0 obeys 




+ C 2 




■ . , - 

-I- mm £{,, —s/mb 


where ak {G)i is the li-norm of the error incurred by approx¬ 
imating 9 with its k largest-magnitude components, mB and 
mb are the number of rows of matrices and 

respectively and the constants are 


^ ^ 1 - (1 - s/252k) _ \/l + S2k 

° l-(l + V2S2k) ' l-(l + V2S2k)' 


Proof: We are interested in deriving a bound 
for ||d ||2 = ll^s ~ know that 9 G 

{C 2 (£B,if,h)nC 2 (£6,L,i)nCoo(^,iT,h) nCoo 
(see Sec. |IV] for notation). Moreover, the solution to 0 is 
either 9 or one with lower li norm, so that we can say that 
||®s||]^ < ll^lli- Using Il23l Lemma 1.6] we know that 


||d ||2 < Co 


CTk {d)l 
s/k 


+ Ui 


I (AdA, Ad) I 

l|dA ||2 


where Ci = , A = Ag U Ai, being Ag the set of 

the k components with largest magnitude of 9 and Ai the set 
of the k components with largest magnitude of dAg (subscript 
denotes restriction to the components indexed by the subscript 
set). Notice that 


||Ad ||2 = 


A9s - A9\\ = IIA0S - y + y - A9\ 




r y(^) 1 





r y (^) 1 


■ ■ 




$( 6)^0 

2 


y(B) 

+ 



-I- $(B)vl/ 0 _y(S) _|_ _ y( 6 ) 


A, 


< 2 min < £b, —r— r + 2 min < £{,, —^Jmf 


Af, 


Using the Cauchy-Schwarz inequality we write 


|(AdA,Ad)| < ||AdA|| 2 ||Ad ||2 
< 2||dA||2\/l + ^2fe ^min|£B, ^,/^|-fmin |£f„ ^ 


Thus, 


|d||2<Co- 


O k (^)i 

\/k 


-I- C 2 min < £b, -—-s/ruB > -f min < £{,, -—s/^ 


Af, 


Next, we characterize the performance of an oracle side 
decoder for sparse signals so that the result can be used to 
provide an oracle optimality condition for the optimization 
of the CS-GQ parameters, as shown in Sec. |VT] The oracle 
decoder is an ideal decoder that knows perfectly the support 
S of the sparse signal. For such systems, recovery amounts 
to computing 9 = A^y, where A^ is the Moore-Penrose 
pseudoinverse of matrix A restricted to the columns indexed 
by S. Often, the oracle receiver is used to evaluate the exact 
performance of CS reconstruction algorithms, since CS decod¬ 
ing involves a nonlinear reconstruction step, whose distortion 
performance is hard to characterize exactly. 

Theorem 2. (Oracle recovery) Suppose that 9 is k-sparse 
and that $ has i.i.d. Gaussian zero-mean entries having 
variance —. Let y = be the vector of measurements to be 

quantized using staggered low- and high-resolution quantizers. 
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Suppose the oracle decoder is used for CS recovery. Then, the 
expected side distortion in the high-rate regime is: 


E 


Os-9 


kr^ 


(to — fc — 1) 24 


( 2 - 2 S 2 “^*') 


and the expected central distortion is: 


E 


Oc-e 


kr^ 


-)-2S 


2 B +1 - 2 ^ + 1 


Proof: According to 
relation holds: 


(to - fc - 1) 24 2^ + 2^ - 1 

Theorem 5], the following 


E 


\e-e\\ 


k 


TO — fc — 1 


E [e"^e] 


(14) 


where e = Q(y) — y is the error introduced by quantization 
Q(-) In case of side decoding, the receiver has only a single 
description i with rriBy high resolution measurements and 
TOb i low resolution measurements. The average error norm 
is obtained as the mean of the norm of the expected error 
when description 1 is received (ei) and when description 2 is 
received ( 02 ). Notice that mB,i = 'rnb ,2 and mb,i = mB ,2 by 
design. Hence, in the high-rate regime the following relation 
holds: 


E 


Os-9 


1 = -^■ 5 (E[oreJ+EKe,]) 

2 ] m — k — 12^'- ■' 

= K , ^(mB,+mbp2-^^ 
TO — fc — 1 24 V 

+ TOb,22 “^® + TOb_22“^^^ 

_ kr^ TO /^-2B I 9-26-1 

(to - fc - 1) 24 *' ’ 


The central decoder exploits the staggering of the low and high 
resolution quantizers to obtain a non-uniform central quantizer, 
having 2^ — 2^ -f 1 bins of size As and 2(2^ — 1) bins of size 
as explained in Sec. m Thus the expected error norm is: 


E [e-* ej = 

TO 


2 B + 2 ^ - 1 


2 (2^ - 1) ^ -p (2^ - 2^ -P 1) ^ 
^ ^ 48 ^ ^12 


mr 


■)-2S 


2 B-El_ 2 b_pl 


24 2^ -P 2^ - 1 

Substituting it into (fl4l l we obtain the desired expression. 


(15) 


VI. Rate-distortion optimization and 

PACKETIZATION 

A. Optimizing CS-GQ 

In this section, we focus our analysis on the scenario when 
one employs identical channel models for both descriptions, 
e.g., the loss probability p is the same for both descriptions. 
Under this scenario, balanced descriptions are optimal both 
instance-wise and on average, i.e. the distortion incurred by 
either side decoder is the same. On the other hand, unbalanced 
descriptions provide the same average performance, but the 
distortion incurred by a specific instance depends on which 
description is received and can be either lower or higher than 
the distortion in the balanced case. Hereafter, we thus only 
consider the balanced case. Different values of the parameters 


B and b provide different tradeoffs between the distortion at 
the central decoder and the distortion at the side decoder, 
for a fixed total rate for both descriptions: R = B -\- b. 
Moreover, the packet loss rate on the communication links 
will also affect the expected distortion. On the other hand, 
the limit case fc = 0, which we called CS-SPLIT, simply 
splits the measurements into two sets without inserting any 
redundancy. This is equivalent to the segmentation performed 
by any network protocol when the packet size exceeds the 
maximum size. It is clear that this is the optimal strategy 
when there are no packet losses, because it is equivalent to 
generating a single description and no bits are wasted in 
redundancy. However, if one description fails to reach the 
receiver, only half of the measurements will be available for 
decoding, hence significantly degrading the quality or leading 
to recovery failure when their number is too low. Fig. |4] shows 
the various operating points enabled by CS-GQ on the side 
distortion vs. central distortion plane. 

It is of interest to study the optimal value of the parameters 
B and 6 when the description loss probability is known. In 
this scenario, one wishes to find the quantization step sizes 
providing the lowest expected distortion for a given bit budget. 
Hence we define the average distortion as 


D=p^ + 2p{l-p)-Ds + {l-pf ■ D, 


where p is the probability of losing a description, and Ds and 
Dc are the distortions incurred by the side and central decoder, 
respectively defined as 


Ds 


ll ^-%||2 

\\0h 


and Dc 


WO-Och 

WOh 


The main problem in the optimization of the parameters is 
the lack of closed-form expressions for the distortion of the 
central and side decoders. There are two possible ways of solv¬ 
ing this problem. The first way is to resort to an operational 
curve, in which the operating points on the {Ds,Dc) plane are 
known in advance (e.g., as a result of experiments). 

In this case, for a fixed total rate R, every choice of the 
low-resolution rate fc generates a {Ds{b), Dc{b)) point, for a 
total of [-jJ -f 1 points. The optimal choice of fc is therefore: 


fc = arg min 
6e[o.f] 


[/ + 2p • (1 - p)Ds{b) -f (1 - pf ■ Dc{b)] 

(Operational Optimality) 


The other possible method is to employ some bounds on 
the reconstruction performance. We can use the performance 
of the oracle decoder, as described in Theorem |2] to derive 
a simple expression that does not require any operational 
information and can be used to optimize the parameters a 
priori. Hence, the minimization of the average distortion 
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Fig. 4: Operating points for CS-GQ (different values of B and b) and for CS-MDSQ (different values of M, refer to HU). 
n = 256, k = 10. 


becomes: 


b = arg min 
&g[o,-S] 

= arg min 
6 e[o.-a] 


+ 2p {1 - p) ■ Ds + {1 - p) ■ Dc 


2p{l-p) 


kr^ 


m 


+ (i-pr 


(m — k — 1) 24 


kr^ 


( 2 - 2 S + 2 - 2 ^) 


■)-2B 


= arg mm 

ftG[0,-|] L 


2^+1 _ 26 + 1 

(to- fc-1)24"' 2® -f 2'’ - 1 

2p (2-2^ -f 2-2'') 

,_ 2 b 2 ^+ 1 - 2 ^ + i‘ 


+ (l-p) 2 - 


= arg min 
6 e[o.-f] L 


2 B + 2 '' - 1 

2p 

2R-b+i _ 2 ^ + 1 


-f (1 -p)2-2(«-'')- 


2R-b + 2 '' - 1 

(Oracle Optimality) 


Exhaustive search over the feasible integer values of b can 
be employed to determine the optimal one. We remark that this 
search just involves evaluating the distortion expression for 
[■jJ -f 1 values of b (e.g., 5 values when R = 8), thus having 
very low complexity. Unfortunately, the optimality would only 
hold if we had an oracle decoder, so this choice might be 
suboptimal. Section IVII-AI shows a comparison between the 
average distortion obtained with operational optimization and 
with oracle optimization in order to show the effectiveness of 
the latter. 
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Fig. 5: Segmenting a long measurements vector. 


B. Packetization for high number of measurements 

It may happen that the MDC system needs to deal with a 
large scale problem in which the number of measurements to 
be acquired is fairly large. In this scenario, the transmitter 
cannot create packets that are arbitrarly large as their size 
is capped by the MTU (Maximum Transmission Unit) size 
of the adopted communication protocol. As an example, the 
IEEE 802.15.4 protocol, popular in sensor network applica¬ 
tions, specifies an MTU equal to 127 bytes with an effective 
maximum payload size of 104 bytes. 

As far as CS-GQ is concerned, a careful segmentation 
operation must be performed whenever the size of a de¬ 
scription exceeds the MTU. The goal is to keep the packets 
balanced, in order to always get useful information whenever 
a packet is received. Fig. |5b] graphically shows a way to 
assign quantization step sizes and performing segmentation. 
Every packet contains an alternating sequence of measure¬ 
ments quantized with B bits and b bits and has a dual packet 
with the opposite alternation pattern. Therefore, every received 
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packet is informative, in the sense that it is able to improve 
the current set of measurements, whether by adding new 
measurements or by improving their accuracy. To fix ideas, 
Fig.|5a]shows a wrong way to perform segmentation, where the 
packets containing only low-resolution measurements are not 
informative if the high resolution dual set is already available. 
In case of loss of a packet, one could potentially lose more 
high resolution measurements with respect to the case depicted 
in Fig. |5b] 

VII. Numerical experiments and applications 

Numerical simulations have been performed to evaluate the 
performance of the proposed technique against other methods 
to implement MDC for CS and to show the advantages for 
some practical usage scenarios, inspired by sensor network 
applications. All the results requiring side decoding have been 
obtained using the side decoder presented in Sec. HYI We 
remark that the algorithm solves problem da and that the 
solution returned by the proposed algorithm is the same as 
the one returned by a solver of convex problems such as 
CVX. However, the proposed algorithm is specialized for the 
particular problem considered in this paper and can be sig¬ 
nificantly faster than CVX. The SeDuMi solver employed by 
CVX uses second-order methods, such as Newton’s method, 
and the complexity per iteration grows as 0{n^). ADMM 
on the other hand is a first order method, thus much less 
expensive per iteration than a second order method, although 
in principle having slower convergence. Nevertheless, ADMM 
enjoys linear convergence rates under mild conditions on the 
cost function, as shown in 1391 . As a practical example, solving 
a problem with n = 1024, k = 150 non-zero components, 
m = 600 measurements obtained by a Gaussian sensing 
matrix, B = 6 bits, b — A bits requires 104 seconds using 
CVX, while only 9.1 seconds are needed by the proposed 
algorithm based on ADMM, thus achieving a tenfold speed¬ 
up. Moreover, whenever quantization cells are uniform, we add 
to the reconstruction problem an constraint in addition to 
the £2 constraint presented in ([T]i. This is to enforce consistent 
reconstruction as discussed in Sec. m 

A. Effectiveness of optimization via oracle formula 

We present an experiment that shows the average distor¬ 
tion obtained via optimization of the parameters with the 
operational method and with the oracle method discussed in 
Sec IVI-Al The curves shown in Fig. |6] are obtained for the 
same system parameters ofFig.|4] i.e., m = 120 measurements 
for a n = 256-long, k = 10-sparse signal and a memoryless 
channel with packet loss probability p. It is observed that the 
optimization via the oracle method provides a good estimate of 
the parameters, only yielding slightly higher distortion when 
the packet loss probability is very small. This is due to the 
ideality of oracle decoding, which estimates lower distortions 
than actually achieved by practical algorithms such as li 
minimization. 

B. Comparison with other quantization-based MDC schemes 

In this section we compare the performance of CS-GQ 
against other methods generating multiple descriptions using 



Fig. 6: Average distortion over a memoryless lossy channel 
with packet loss probability p. Parameters b and B are opti¬ 
mized via oracle/operational formulas, n = 256, m = 120, 
k = 10. 


quantization. Thus, we compare CS-GQ, CS-SPLIT and CS- 
MDSQ, where CS-MDSQ is an application of the multiple 
description scalar quantizer (MDSQ) as introduced in IfTTII 
to the quantization of compressive measurements. Although 
MDSQ is a powerful and popular solution for MDC, and 
provides asymptotic performance close to the rate-distortion 
bound, its use in CS is not straightforward. In fact, CS recon¬ 
struction is a nonlinear process, with no guarantee that lower 
distortion on the measurements results in better reconstruction 
quality. Hence, the choice of the number of measurements and 
quantization step size is a trade-off in order to achieve the 
best performance after recovery. In our simulations, we use 
the index assignment technique developed in ifT^ . As a first 
comparison we fix the number of measurements and obtain 
the tradeoff points on the side/central distortion plane for both 
CS-GQ and CS-MDSQ. In Fig. |4] it is observed that fixing 
m = 120 for a n = 256-long, k = 10-sparse signal allows CS- 
GQ to provide better tradeoff points than CS-MDSQ. However, 
CS-MDSQ outperfroms CS-GQ when a lower number of mea¬ 
surements is chosen. We remark that, theoretically, CS-MDSQ 
allows 2^+^ tradeoff points by modulating the redundancy, 
while CS-GQ admits [-jJ -P1 points only. However, observing 
Fig. a we can notice that the redundancy of many tradeoff 
points of CS-MDSQ is so high that the resulting side distortion 
is severe (e.g., above 0.4). We remark that the MDSQ can also 
accomodate the zero-redundancy case when all the entries of 
the index assignment matrix are used, corresponding to CS- 
SPLIT. Both zero-redundancy CS-MDSQ and CS-SPLIT ob¬ 
tain the same central distortion. However, side distortion may 
be different. CS-SPLIT represents half of the measurements 
with a fine quantization rate, while CS-MDSQ represents all 
the measurements with a coarser quantization rate. The effect 
on CS recovery is that when there are plenty of measurements 
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(depending on n and on the sparsity of the signal), the 
side distortion after recovery is quantization-limited, so even 
though CS-MDSQ provides more measurements, their coarser 
quantization limits the performance. We also notice that the 
full-redundancy case (B = b) corresponds to a CS-MDSQ 
with staggered side quantizers and equal size intervals in the 
central partition. 

A further experiment evaluates the dependency of the side 
and central distortion on the number of measurements. Hence, 
we fix the number of measurements and an operating point 
and we analyse how central and side distortion change as a 
function of the number of measurements. Fig. [T] is obtained 
with the same signal as in Fig. 0] and choosing m = 120, 
{B, b) = (6, 2) as the operating point of CS-GQ and M = 2 as 
the redundancy of the MDSQ with R = 8 total rate (namely, 
the ratio between the step size of the side quantizer and of 
the central quantizer is 2M; refer to m for further details). 
We observe that the distortion of CS-GQ decreases faster than 
the distortion of CS-MDSQ as the number of measurement 
increases, while central distortion decreases at the same rate 
although CS-GQ is marginally better. The reason behind this 
performance is that CS-MDSQ is more quantization-limited 
than CS-GQ when the number of measurements is high. 
Viceversa, for a very low number of measurements, CS-GQ 
is measurement-limited. The previous experiment considered 
a rather generous overall budget equal to Rm = 960 bits. 
Indeed, we considered a hxed number of measurements and 
chose the rate according to the budget. It must be noticed 
that one could optimize both the value of m and the value of 
R under the overall budget constraint and this could lead to 
different choices for CS-GQ and CS-MDSQ. Although we do 
not report the results for brevity, we observed that if the budget 
is very large as in the previous case, CS-MDSQ generally 
provides better tradeoff curves than CS-GQ by using fewer 
measurements and a finer quantization rate. However, results 
change under a tight budget constraint. A minimum number of 
measurements has to be acquired in order to ensure successful 
reconstruction and due to the tight budget constraint it is not 
possible to use high quantization rates. Figl^ compares the 
tradeoff curves of CS-GQ and CS-MDSQ for the two operating 
points m = 50 and R = 8 and for m = 100, R = A. Those 
two choices of m and R are optimal for CS-MDSQ and CS- 
GQ respectively under a budget of Rm = 400 bits. We notice 
that CS-GQ outperforms CS-MDSQ despite choosing the best 
combination of m and R. We therefore conclude that CS-GQ 
can be advantageous with respect to CS-MDSQ when the bit 
budget is low. 

C. Simulations over MTU-limited memoryless and Gilbert 
channels 

In this section we perform some simulations to assess the 
gain achieved by using CS-GQ when the communication chan¬ 
nel is prone to packet loss. We consider two channel models, 
which are signihcant for the performance assessment of CS- 
GQ in a packetization scenario: the memoryless channel, in 
which the loss trace is a Bernoulli process, and the Gilbert 
channel 113, im, where memory is modelled as a two- 
state Markov chain, as shown in Fig. |9l In this model the 




m 

(b) Central Distortion 


Fig. 7: Comparison of CS-GQ and CS-MDSQ. {B, b) = (6, 2) 
for CS-GQ and M = 2, R = 8 for CS-MDSQ. n = 256, 
k = 10. 


communication link can be in either of two states, that we 
label Good (G) and Bad (B) with a probability p of moving 
from G to B and probability q of moving from B to G. When 
in B state the link will drop any packet transmitted. This is a 
popular model for channels exhibiting burst errors, e.g., due 
to fading in a wireless system. In both cases we assume that 
a limit to the maximum packet size is present and it is equal 
to 100 bytes, as in the MTU of IEEE 802.15.4. 

The simulations over the memoryless channel have been 
performed with a signal of length n = 256, A: = 40 non-zero 
components, m = 160 measurements and a total rate i? = 10 
bits per measurement over the two descriptions. CS-GQ has 
been implemented to create two packets of 100 bytes each, in 
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Fig. 8: Comparison under a tight budget Rm = 400 bits. 
Optimal choice for CS-MDSQ is m = 50,i? = 8. Optimal 
choice for CS-GQ is m = 100,i? = 4. 



q 


Fig. 9: The Gilbert-Elliot channel model, p is the transition 
probability from good to bad state. 


the same fashion of Fig. |5b] and compared against a standard 
segmentation of the measurements, i.e., splitting half of the 
measurements into the two packets of 100 bytes. The tests 
measured the distortion in the reconstructed signal averaged 
over 10® runs. The values of B and b are automatically 
optimized using the oracle method (see Sec. IVl-Al l. Fig. [TO] 
shows that CS-GQ has superior performance, yielding a lower 
average distortion than segmentation of the measurements and 
the gain is more significant when the packet loss probability 
is high. 

The simulations over the Gilbert channel have been per¬ 
formed using a longer signal, in order to correctly test the 
effect of the memory of the channel. We suppose that a batch 
of Ns = 1000 vectors of measurements have to be transmitted 
in sequence. The length of the signals is n = 1000, k = 200 
non-zero components, m = 720 measurements and a total 
rate i? = 10 bits per measurement. Table |I] reports the average 
distortion for some values of the transition probabilities p and 
q. It can be noticed that the proposed scheme allows to achieve 
lower distortion with respect to the segmentation approach. 


D. Simulations of object recognition over lossy channel 

In this section we consider an application recently proposed 
in ll42ll and ll43ll as a possible scenario in which the techniques 
proposed in this paper could provide a significant performance 


Fig. 10; Simulations over memoryless channel. 


TABLE I: Average distortion over the Gilbert-Elliot channel 


p 

q 

Segmentation 

CS-GQ 

0.05 

0.5 

0.0257 

0.0192 

0.05 

0.3 

0.0762 

0.0496 

0.05 

0.15 

0.1820 

0.1205 

0.01 

0.3 

0.0160 

0.0102 

0.01 

0.15 

0.0451 

0.0324 


improvement. The purpose is to show the effectiveness of CS- 
GQ in a practical scenario. In this application, a smart camera 
platform, such as CITRIC ll44l . computes histograms of image 
descriptors to be used as features in a scene classification task. 
Image descriptors are compact representations of an image 
that provide a certain degree of invariance to transformations 
such as rotation, scaling, etc., and are widely used in the 
computer vision field for visual search or scene recognition 
tasks. The most popular descriptors are the Scale-Invariant 
Eeature Transform (SIET) ll45l . which describe each image 
keypoint, i.e., a signficant point of the image to be included in 
the descriptor, through a vector of s = 128 entries. Each image 
i has a variable number Ni of keypoints, and thus associated 
descriptors. A clustering algorithm, such as K-means, is used 
to identify k clusters in K® from the Ni original descriptors. 
A histogram of descriptors is obtained by counting how many 
keypoint descriptors fall in each cluster. In ll4^ . the authors 
leverage the sparsity of such histograms to compress them 
by means of random projections, and transmit them to a 
remote fusion center. The authors do not consider the problems 
of quantization of the measurements, nor the possibility of 
having channel losses. However, we remark that those are 
key problems for this application. In fact, transmitting the 
random projections as floating point values requires very large 
bandwidth, hence quantization could reduce such requirement 
and save a sizeable amount of the scarce energy of the mobile 
platform. Also, channel losses may occur and retransmission 
of packets may not be feasible due to real-time constraints, 
and it would indeed require further energy consumption on 
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TABLE II: Accuracy of scene recognition 


p 

Segmentation 

CS-GQ 

0.01 

70.63% 

71.25% 

0.05 

61.25% 

63.75% 

0.10 

54.38% 

61.88% 


sensor side. Hence disabling retransmission and providing a 
scheme that is robust to channel losses could be of interest for 
such platforms. 

In our tests, we build a 10,000-dimensional dictionary by 
running hierarchical K-means, with 10 clusters per level in the 
hierarchy, over SIFT descriptors extracted from some images 
in the BMW database ll42l . This database comprises photos of 
the UC Berkeley campus acquired with the CITRIC platform 
with multiple views of the same scene from different angles. 
We then use the dictionary of quantized SIFT features to 
produce histograms of descriptors. Each image is characterized 
by a histogram of descriptors, which is sparse because the 
number of descriptors extracted from each image is much less 
than the number of entries in the dictionary. na. A support 
vector machine is trained from histograms of descriptors of the 
images in the training set, in order to solve a classihcation task 
that amounts to recognizing a scene from a number of possible 
scenes. In order to simulate a real system, we consider packets 
with maximum size of 100 bytes (as in the case of the MTU of 
IEEE 802.15.4), to be transmitted over a memoryless channel. 
The test metric that we adopt in this case is the accuracy in 
the scene recognition. Table HJ shows the accuracy in the scene 
recognition task as function of the packet loss probability p. 
We can see that CS-GQ builds robustness into the system and 
allows to achieve higher recognition rates, when in presence 
of packet losses. 

VIII. Conclusions 

In this paper we considered the problem of robust trans¬ 
mission of CS measurements over unreliable channels. We 
discussed some strategies based on the Graded Quantization 
paradigm that enable to increase robustness in a way that 
directly exploits the democratic property of compressive mea¬ 
surements. Moreover, contrary to many cases in the literature, 
we considered a realistic packetization scenario and discussed 
the issues involved in it as well as the performance of the 
proposed method. Finally, we proposed a novel reconstruc¬ 
tion algorithm, based on ADMM, for the specihc problem 
of having measurements obtained from multiple quantizers. 
Our simulations showed that the proposed technique is a 
competitive method to implement MDC for CS applications. In 
particular, we showed that we can improve scene recognition 
accuracy in an application of a smart camera platform using 
CS and transmitting over an unreliable channel. 
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